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VOLUME  I 


COST  DATA  ANALYSIS  METHODOLOGY 
FOR 

THE  DEFENSE  NUCLEAR  AGENCY/MULTI-AGENCY  COOPERATIVE 
EMP  HARDENING  TECHNOLOGY  PROGRAM 

I.  INTRODUCTION 


The  Defense  Nuclear  Agency/Multi-Agency  Cooperative  EMP  Har¬ 
dening  Technology  Program  will  result  in  a  variety  of  EMP  protec¬ 
tion  designs  and  validation  procedures  for  strategic  aircraft 
systems.  Using  a  C-13Q  "test  bed"  and  certain  laboratory  facili¬ 
ties/  the  program  will  test  alternative  EMP  protection  concepts 
and  designs.  Technical  and  cost  data  will  be  generated,  col¬ 
lected,  and  evaluated.  These  data  will  be  used  to  plan  EMP 
retrofit  programs  for  existing  aircraft,  and  to  plan  the  EMP  pro¬ 
tection  of  new  aircraft. 

Technical  data  will  include  design  guidelines,  baseline 
tests,  flight-certification  of  hardware,  and  test  and  validation 
procedures.  These  data  will  be  made  available  co  Air  Force  Spe¬ 
cial  Project  Offices  in  a  documentation  format  used  by  the  air¬ 
craft  industry.  The  technical  data  base  will  be  developed 
throughout  the  program,  which  is  expected  to  continue  through 
fiscal  year  1988. 

In  a  similar  fashion,  cost  data  will  be  generated,  collected, 
and  evaluated  throughout  the  program.  If  sufficient  detail  is 
achieved  in  the  cost  data  base,  EMP  protection  costs  can  be 
extrapolated  for  a  wide  variety  of  aircraft  systems. 

A  life  cycle  cost  model  should  be  set  up  to  provide  a  metho¬ 
dology  for  collecting  cost  data.  The  model  should  include  the 
following  cost  categories:  program  management,  requirements 
definition,  protection  design,  fabrication,  installation  and 
checkout,  test  and  evaluation,  and  finally,  protection  operations 
and  maintenance. 

The  life  cycle  cost  model  should  contain  three  special 
features.  First,  it  should  separate  costs  that  are  unique  to 
this  particular  program  from  costs  that  will  be  incurred  in 
future  aircraft  EMP  protection  programs.  This  separation  will 
permit  future  users  of  the  data  base  to  extract  only  those  cost 
elements  which  pertain  to  their  respective  programs.  Second, 
this  model  should  permit  detailed  adjustments  for  inflation. 
Inflation  indices  can  be  used  to  update  particular  labor  and 


material  costs.  Third,  learning  curve  analysis  should  be  applied 
to  labor-intensive  activities.  If  EMP  protection  retrofit  pro¬ 
grams  are  implemented  on  a  large  scale,  learning  curve  analysis 
would  be  appropriate  to  enhance  the  accuracy  of  forecasts  for 
certain  labor  costs. 

To  achieve  this,  cost  categories  should  be  broken  down  into  a 
number  of  sub-categories.  For  example,  the  requirements  defini¬ 
tion  category  should  include  sub-categories  ior:  facility  pro¬ 
tection  requirements,  facility  survey  costs,  requirements 
analysis,  and  surveillance  and  maintenance  requirements.  Each 
sub-category  should  include  elements  which  indicate  specific 
labor,  materials,  equipment,  travel,  and  overhead  charges.  For 
example,  the  facility  protection  requirements  sub-category 
includes  elements  for  scientific  and  consultant  manhours,  design 
engineering  manhours,  pre-installation  test  and  evaluation 
manhours,  design  quality  control  manhours,  test  and  evaluation 
equipment  costs,  travel  and  design  overhead  costs. 

At  this  point,  it  is  also  pertinent  to  consider  how  the  cost 
data  base  might  be  used.  In  my  judgement,  program  management, 
requirements  definition,  design,  fabrication,  and  installation 
and  checkout  are  essentially  research  and  development  (R&D) 
activities.  As  shown  by  W.J.  Weida,  an  S-curve  with  cumulative 
R&D  dollars  spent  as  the  dependent  variable  and  time  as  the 
independent  variable  provides  a  highly  accurate  model  for  fore¬ 
casting  R&D  expenditures  per  time  period.  The  problem  remains  of 
what  to  do  with  the  test  and  evaluation  and  the  operations  and 
maintenance  cost  data.  Under  certain  conditions,  we  may  be  able 
to  use  all  life  cycle  cost  categories  in  a  benefit-cost  model. 

The  remainder  of  this  paper  provides  details  on  a  life  cycle 
cost  model,  an  S-curve  R&D  cost  forecasting  model,  and  a 
benefit-cost  model  for  comparing  alternative  EMP  protection 
designs.  The  life  cycle  cost  model  is  based  upon  the  work  of' 
Booz-Allen  and  Hamilton,  Inc.  and  myself  for  the  DNA  Life  Cycle 
Cost  Experiment.  The  S-curve  model  can  be  developed  from  the 
cost  data  base  with  no  limiting  requirements.  The  benefit-cost 
model  has  one  critical  requirement:  it  must  include  some  measure 
of  the  relative  benef.its  of  the  alternative  designs.  If  this 
measure  is  achieved,  we  can  legitimately  use  a  benefit-cost  model 
to  compare  the  alternative  EMP  protection  designs.  Without  such 
a  measure,  legitimate  life  cycle  cost  comparisons  cannot  be  made. 


II.  LIFE  CYCLE  COST  MODEL 


The  following  proposed , Life  Cycle  Cost  Model  is  essentially  a 
well-ordered  data  base.  Its  categories,  sub-categories,  and  ele¬ 
ments  permit  the  identification  of  unique  costs  for  the  C-130 
testbed  program,  detailed  adjustments  for  inflation,  and  learning 
curve  analysis  for  labor-intensive  activities.  It  requires 
inputs  of  cost  data  from  program  managers,  consultants,  EMP  pro¬ 
tection  contractors,  test  and  evaluation  contractors,  and  opera¬ 
tions  and  maintenance  agencies.  Cost  data  should  be  submitted  on 
a  quarterly  basis  to  a  designated  collection  agency.  Outputs  of 
the  model  will  Include  the  cost  data  base,  identification  of  cost 
drivers,  and  guidelines  for  future  users.  Finally,  the  model 
will  provide  a  data  base  for  a  budget  forecasting  model  and  a 
benefit-cost  model. 
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NOTES:  1  =  Scientific  &  Consultant 

2  ^  Test  6  Evaluation 

3  =  Quality  Control 


III. 


S-CURVE  R&D  COST  FORECASTING  MODEL 


Background 

As  indicated  earlier,  W.J.  Weida  has  shown  that  an  S-curve  is 
a  highly  accurate  form  for  forecasting  R&D  dollars  spent  per  time 
period.  If  we  realize  that  an  S-curve  is  merely  the  cumulative 
form  of  a  bell-curve  (which  may  or  may  not  be  skewed)  as  shown  in 
the  following  figure,  a  methodology  for  forecasting  R&D  costs  per 
time  period  becomes  apparent. 


Figure  1:  Derivation  of  the  S-Curve 

The  S-curve  can  be  fitted  to  historic  cost  data  on  similar 
R£D  projects.  Essentially,  two  curves  are  fitted  to  the  data, 
one  curve  from  time  zero  to  the  inflection  point,  and  the  other 
curve  from  the  inflection  point  to  the  data  point  obtained  at  the 
end  of  the  R&D  effort.  Both  fitted  curves  follow  a  quadratic 
form. 

y  ■  a  +  b^x  +  b2x2, 

where  a,  b-^,  and  b2  are  coefficients  estimated  from  a  least 
squares  regression.  Y  and  x  are  the  dependent  and  independent 
variables,  respectively.  Before  fitting  the  two  curves,  the  data 
are  normalized  to  percent  cumulative  dollars  expended  (y)  and 
percent  time  expended  (x) .  Normalization  permits  the  use  of  the 
S-curve  to  forecast  R&D  costs  for  programs  that  differ  in  total 
dollars  spent  and  time  expended.  The  inflection  point  can  easily 
be  calculated  by  looking  at  the  second  differences  of  cost  with 
respect  to  time,  and  the  two  curves  can  be  joined  at  this  point. 
Standard  confidence  interval  techniques  can  be  used  to  assess  the 
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variation  of  actual  K&D  costs  from  forecasted  costs.  The  fore¬ 
cast  can  be  updated  as  data  become  available  after  a  new  R&D 
effort  is  started. 

Developing  and  Using  the  S-Curve 

The  general  method  for  developing  an  S-curve  from  the  C-13G 
testbed  and  cost  data  from  other  aircraft  EMP  protection  programs 
can  be  described  in  a  five-step  procedure. 

Step  1.  R&D  costs  per  time  period  from  the  C-13Q  testbed 
program  and  R&D  costs  per  time  period  from  other  aircraft  EMP 
retrofit  programs  should  be  gathered  and  recorded  as  a  cumulative 
percentage  of  total  R&D  expenditure.  These  R&D  expenditures 
include  all  generic  cost  data  gathered  in  the  program  management, 
requirements  definition,  design,  fabrication,  and  installation 
and  checkout  categories.  Similarly,  the  amount  of  time  over 
which  the  R&D  effort  for  each  program  occurred  should  be  deter¬ 
mined  and  each  succeeding  time  increment  should  be  recorded  as  a 
cumulative  percent  of  the  total  program  time.  This  step  has  the 
effect  of  normalizing  the  data  for  all  programs  considered  for 
the  data  base.  See  Figure  2. 

1,  of 

£*?&**&  t  I 
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Figure  2;  Expenditures  vs.  Time 


Step  2.  With  the  data  arrayed  in  a  normalized  format  and 
plotted  on  the  axes  of  Figure  2,  the  budget  expenditure  pattern 
may  be  immediately  checked  for  general  conformity.  This  is 
accomplished  by  determining  whether  or  not  the  cumulative  expen¬ 
diture  curve  follows  the  S-curve  pattern  established  by  Weida  for 
all  previous  Department  of  Defense  R&D  projects,  i.e.,  if  air¬ 
craft  EMP  protection  retrofit  program  cumulative  budget  expendi¬ 
tures  follow  the  pattern  shown  in  Figure  3,  then  these  expendi¬ 
tures  are  in  accordance  with  past  R&D  experience. 


"7*  TarrtL. 
rtf^e 


Figure  3:  The  General  R&D  S-Curve 

According  to  Weida,  the  general  curve  can  be  described  by  the 
following  equations: 

Y  =  -0.0124  +  0.5376X  +  1.396X2  (Bottom  Half) 

Y  =  -0.5345  +  3.1150X  -  1.504X2  (Top  Half) 


The  one-standard  deviation  (la)  confidence  interval  about  the 
inflection  point  was  described  by  Weida  as  follows: 

Mean:  0.562  (vertical  axis),  0.462  (horizontal  axis) 

=  0.05402  (vertical  axis) 

°x  =  0.07300  (horizontal  axis) 

If  the  expenditure  pattern  does  not  follow  the  general  S- 
curve  pattern,  then  alternative  model  specifications  should  be 
tried.  For  example,  one  might  use  a  logarithmic  form  as  a  means 
to  describe  the  cumulative  expenditure  pattern. 

Step  3.  Next,  locate  the  largest  incremental  change  in  cumu¬ 
lative  expenditures  which  is  followed  by  two  periods  of  decreas¬ 
ing  cumulative  expenditures.  This  increment  is  designated  as  the 
inflection  point.  The  S-shaped  curve  is  broken  at  this  point  and 
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the  inflection  point  becomes  the  last  data  point  in  the  fitst  (or 
lower)  curve  and  the  first  data  point  on  the  second  (upper) 
curve.  This  common  point  allows  the  curves  to  be  spliced  again 
after  curve  fitting.  The  mean  inflection  point  and  1(7  values  for 
the  general  S-curve  were  described  earlier?  however,  past  experi¬ 
ence  has  shown  a  high  degree  of  variability  in  the  inflection 
point  locations  compared  to  the  general  S-curve. 

Step  4.  Equations  for  the  lower  and  upper  portions  of  the 
S-curve  are  developed  using  standard  regression  techniques. 
Again,  data  inputs  include  the  normalized  cumulative  costs  for 
the  C-13G  testbed  program  and  the  normalized  cumulative  costs 
from  other  aircraft  EMP  protection  programs  for  which  data  are 
available.  Particular  care  must  be  taken  in  this  step  to  assure 
that  the  curve  equations  which  are  developed  have  dealt  with  the 
problems  inherent  in  the  use  of  time  series  data.  Failure  to 
correct  the  problem  of  autocorrelation  will  result  in  curve  equa¬ 
tions  which  are  of  little  value  and  which  will  adversely  affect 
the  performance  of  the  completed  model.  To  correct  this  problem, 
the  Cochran-Orcutt  procedure  for  alleviating  serial  autocorrela¬ 
tion  is  usually  applied. 

Step  5.  Once  the  curve  equations  have  been  developed  from 
the  budget  data,  two  specific  types  of  knowledge  have  been 
gained.  First,  the  equation  form  which  best  fits  the  R&D  budget 
data  has  now  been  determined.  This  is  usually  a  quadratic  form 
for  both  the  upper  and  lower  halves  of  the  S-shaped  curve.  '  This 
specific  curve  form  should  be  used  with  any  actual  expenditures 
when  later  attempts  are  made  to  forecast  the  R&D  costs  of  future 
aircraft  EMP  protection  programs.  Second,  equations  expressing 
the  subjective  planning  inherent  in  the  R&D  portion  of  an  air¬ 
craft  EMP  protection  program  are  now  available  for  the  upper  and 
lower  parts  of  the  S-shaped  curve.  These  equations,  can  be  used 
as  guides  during  forecasting,  thus  providing  a  method  of  incor¬ 
porating  this  subjective  information  into  the  final  cost  forecast- 
for  a  future  program. 


The  S-Curve  as  a  Forecasting  Tool 


The  methodology  de.veloped  in  the  previous  section  will  result 
in  an  S-curve  for  forecasting  R&D  costs  for  future  aircraft  EMP 
protection  programs.  It  is  appropriate  at  this  point  to  convey 


The  methodology  followed  in  this  section  is  an  abbreviated 
version  of  Weida's  presentation  in  A  General  Technique  of  R&D 
Forecasting ,  U.S.  Air  Force  Academy  Technical  Report  77-12,  Sep¬ 
tember,  1977. 


the  proper  method  for  employing  the  S-curve  as  a  management  tool. 
The  program  manager  should  view  the  forecast  as  a  non-threatening 
means  of  alerting  managers  to  possible  program  difficulties  and 
it  should  be  presented  not  as  a  point  estimate,  but  rather  as  a 
range  of  values  within  which  the  end  cost  of  the  program  is 
likely  to  fall  if  the  present  courses  of  action  are  continued. 
For  the  purpose  of  this  paper ;  three  points  along  this  possible 
range  of  cost  will  be  identified  as:  (1)  the  best  possible  R&D 
cost,  (2)  the  most  likely  R&D  cost,  and  (3)  the  worst  possible 
R&D  cost.  The  best  possible  R&D  cost  occurs  if  the  second  half 
of  the  program  follows  exactly  the  R&D  curve  irrespective  of  the 
performance  record  established  in  the  first  half  of  the  program. 
The  most  likely  program  cost  is  obtained  if  the  second  half  of 
the  program  follows  the  course  indicated  by  the  R&D  curve  as 
updated  by  data  made  available  from  the  first  half  of  the  air¬ 
craft  EMP  protection  program  being  forecasted.  The  worst  possi¬ 
ble  program  cost  would  be  indicated  by  the  upper  limit  of  the 
confidence  interval  around  the  updated  forecast. 

These  three  types  of  forecasts  are  shown  in  Figure  4.  The 
details  involved  in  forming  each  of  these  forecasts  will  now  be 
discussed. 
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Figure  4:  Three  Possible  Forecasts 


The  Best  Possible  Cost 


First,  derive  the  two  halves  of  the  equation  for  the  S-shaped 
curve  in  the  manner  previously  outlined.  This  gives  curve  1  of 
Figure  5,  the  aircraft  EMP  protection  program  B&D  curve,  or  for 
the  purpose  of  this  discussion,  the  budget  curve. 

Assume  now  that  the  first  data  points  concerning  actual 
expenditure  information  have  become  available.  These  data  points 
are  first  deflated  by  dividing  the  dollar  figures  by  an  appropri¬ 
ate  inflation  index.  Studies  have  shown  that  the  GNP  Deflator  is 
usually  a  good  choice  for  this  index. 1  The  deflated  figures  are 
then  converted  to  percentage  figures  by  dividing  by  the  latest 
deflated  total  program  cost,  and  these  percentage  figures  are 
plotted  on  the  axis  oE  Figure  2.  This  leads  to  the  beginning  of 
an  "actuals-  curve.  These  actuals  may  be  used  to  forecast  a  new 
end  cost  for  the  program  as  follows: 

(1)  Derive  a  new  lower  half  of  the  S-shaped  curve  by 
fitting  the  actuals  to  an  equation  of  the  form  found  to  be 
appropriate  for  the  budget  data — in  general,  this  will  be  a  qua¬ 
dratic  curve. 

(2)  Using  this  quadratic  curve  equation,  inst  v.  the  per¬ 
cent  of  total  time  figure  for  the  budget  curve  inflection  point 
(35%  on  Figure  5)  to  forecast  a  new  inflection  point,  and  then 
use  other  points  on  the  X  (time)  axis  to  derive  a  new  lower  'half 
for  the  S-shaped  curve. 

(3)  Now  take  the  equation  which  was  developed  for  the 
top  half  of  the  budget  curve  and  substitute  the  percent  time  and 
percent  budget  figures  for  the  forecast  inflection  point  into 
this  equation  to  calculate  a  new  intercept  for  the  upper  curve. 
This  new  intercept,  along  with  the  original  slope  figures  from 
the  budget  curve,  has  the  effect  of  "splicing"  the  equation 


2 

Brush,  John  S.,  “Study  of  Possible  Improvements  in  the  Ac¬ 
curacy  of  Aeronautical .Economic  Escalation  Indices,"  unpublished 
paper,  USAF  Academy,  Colorado,  February  1976.  Alternatively,  in¬ 
dices  for  specific  labor  and  materials  can  be  found  in:  The  Sta¬ 
tistical  Abstract  of  the  United  States ,  published  annually  by  the 
U.S.  Department  of  Commerce.  Sections  12,  14,  and  16  are  of  par¬ 
ticular  interest  in  the  1984  edition.  Another  source  of  special¬ 
ized  indices  would  be:  Basic  Economic  Statistics ,  published 
monthly  by  the  Bureau  of  Economic  Statistics,  Inc.,  Washington, 
D.C.  Part  1  is  of  particular  interest  in  the  March  1984  edition. 
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developed  from  the  first  halt  actuals  to  the  budget  equation  Cor 
the  second  half  of  the  curve;  all  of  which  yields  the  new  S- 
shaped  curve  4  of  Figure  5.  In  addition,  this  procedure  allows 
the  development  of  a  forecast  for  the  end  cost  of  the  project 
which  is  constrained  by  the  planning  and  other  subjective  infor¬ 
mation  inherent  in  the  original  budget  curve. 


%  Torni. 


(4)  At  this  point,  a  program  manager  may  take  several 
different  approaches.  First,  if  he  wants  to  learn  the  absolute 
figure  for  the  final  cost  of  the  project,  curve  4  may  be  modified, 
by  inclusion  of  inflation  data.  In  this  case,  the  forecast 
expenditure  data  of  curve  4  would  be  multiplied  by  an  inflation 
index  to  get  a  new  curve  which  is  labeled  5  in  Figure  5.  How¬ 
ever,  in  doing  this  he  should  have  in  mind  a  concept  of  the 
errors  inherent  in  any.  process  such  as  the  one  just  described. 

Up  to  this  point  we  have  not  mentioned,  for  the  sake  of  sim¬ 
plicity,  that  there  is  an  error  involved  in  forecasting  which 
should  be  expressed  as  a  confidence  interval  around  curve  4.  The 
confidence  band  indicates  that,  with  some  given  probability,  one 
may  expect  the  real  value  for  any  point  on  the  line  to  fall  some¬ 
where  within  this  particular  interval.  When  the  budget  curve  is 
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compared  with  the  forecast  curve,  only  one  error,  the  standard 
error  of  the  forecast,  must  be  considered.  This  leads  to  the 
situation  shown  in  Figure  6. 
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Figure  6:  The  Error  of  the  Forecast 

Here  the  confidence  band  indicates  the  possible  range  of  values 
(from  b  to  c)  in  which  the  true  cost  of  the  program  is  expected 
to  fall,  and  similarly,  the  range  of  the  size  (from  (1)  to  c]  of 
the  potential  program  overrun. 

However,  if  one  desires  to  compare  the  full  cost,  with  inf la-- 
tion,  of  the  project  (Curve  5,  Figure  5)  with  the  full  inflated 
cost  of  the  budget,  both  the  error  of  the  forecast  and  the  error 
involved  in  developing  the  inflation  figures  must  be  considered. 
This  has  the  effect  of  greatly  increasing  the  size  of  the  confi¬ 
dence  bands  as  is  shown  in  Figure  7.  The  end  result  is  that  the 
ability  to  compare  the  final  cost  of  the  project  with  the  budget 
cost  is  greatly  impaired.  As  Figure  7  shows,  in  this  case  one 
could  anticipate  a  tremendous  overrun  or  an  underrun  [a  -  d]  from 
the  same  data. 
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Figure  7:  The  Error  of  the  Forecast  and 
The  Error  of  the  Inflation  Forecast 

The  lesson  here  is  to  compare  figures  in  a  manner  which  will 
minimize  the  errors  involved  in  the  comparison.  In  other  words, 
the  best  picture  of  the  status  of  a  project  may  be  gained  by  com¬ 
paring  the  two  curves  shown  in  Figure  6.  This  comparison  pro¬ 
vides  all  of  the  information  required  for  day-to-day  management 
of  the  program.  If  a  full  end  cost  of  the  program,  is  desired, 
this  can  be  developed  quickly  by  simple  multiplication  utilizing 
whatever  inflation  forecast  is  deemed  appropriate  at  the  time 


that  the  information  is  required. 

This  does  not  mean,  however,  that  the  program  manager  should 
not  use  the  actual  inflation  data  when  it  is  available.  In  this 
case,  no  errors  of  forecast  are  present  because  the  actuals  in 
both  program  cost  and  inflation  rates  are  known.  This  makes  it 
very  easy  to  remove  the  effects  of  inflation  to  see  how  much  of 
an  overrun  is  actually  attributable  to  other  causes. 

Figure  8  shows  a  case  in  which  the  deflated  budget  curve  1  is 
modified  by  the  actual  experienced  inflation  to  derive  curve  2. 
One  may  readily  compare  this  curve  with  the  contractor's  inflated 
actuals  (curve  3}  to  determine  the  actual  extent  of  the  overrun. 
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Another  situation  which  this  method  of  program  roonito"  1 
will  easily  handle  is  the  case  of  the  schedule  slippage  or 
gram  extension.  Of  the  two,  the  slippage  is  the  most  severe 
because  it  often  occurs  early  in  the  project  where  it  has  s  pro¬ 
found  effect  on  costs.  Assume  once  again  the  basic  deflated 
budget  curve  shown  in  Figure  9  with  an  actual  deflated  expendi¬ 
ture  curve  as  shown. 
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Figure  9:  The  Program  Slippage  Situation 


It  would  appear  upon  initial  inspection  that  the  program  is  run¬ 
ning  slightly  below  the  planned  expenditures  at  time  t.  However, 
it  is  revealed  that  the  R6.D  program  is  actually  behind  schedule, 
having  only  accomplished  the  number  of  milestones  associated  with 
time  t-1.  To  compensate  for  this  slippage,  move  curve  2  back  one 
unit  from  t  to  t-1  so  that  the  actual  expenditures  are  now  shown 
as  curve  3  in  their  proper  relationship  with  the  budget  curve. 
This  is  actually  accomplished  mathematically  by  calculating  a  new 
inflection  point  which  will  reflect  the  slippage  in  the  schedule. 
This  new  point  is  derived  from  the  equation  for  curve  2  by  calcu¬ 
lating  the  inflection  point  not  at  time  T,  the  location  of  the 
original  point,  but  rather  at  time  T  +  1,  the  location  of  the 
inflection  point  after  slippage  has  occurred.  This  new  inflec¬ 
tion  point  becomes  the  intercept  of  the  equation  for  the  top  half 
of  the  budget  curve,  ajid  the  time  values  which  are  used  to  fore¬ 
cast  from  the  top  half  of  the  budget  curve  now  start  at  the  T  +  1 
increment  (instead  of  T)  and  continue  to  the  100%  +  1  increment 
(instead  of  the  100%  increment)  . 


The  M ost  Likely  Cost. 


Forecasting  the  most  likely  cost  proceeds  in  the  same  manner 
listed  in  the  previous  section  up  to  the  point  at  which  a  new 
inflection  point  is  forecast.  The  actuals  are  converted  t >  per¬ 
centages  and  plotted  in  the  same  manner,  and  the  curve  form  to 
plot  those  actuals  is  the  same  equation  type  selected  to  describe 
the  bottom  half  of  the  general  curve.  At  this  point,  however, 
the  method  of  forecasting  changes  considerably. 

Instead  of  merely  splicing  the  top  half  of  the  general  curve 
onto  the  new  bottom  curve,  the  bottom  curve  is  actually  mapped 
into  tiie  general  curve  framework.  This  is  accomplished  as  fol¬ 
low.';  : 

(1)  Using  the  deflated  actuals  from  the  program,  fit 
whatever  curve  form  is  used  in  the  bottom  half  of  the  general 
curve  to  these  data  and  forecast  a  new  value  for  cumulative 
expenditures  at  the  inflection  point.  This  new  value  is  found  by 
substituting  the  cumulative  percent  time  figure  which  corresponds 
to  the  general  curve  inflection  point  into  the  new  equation  which 
v.’.ts  derived  from  the  actuals. 

(I!)  Take  the  new  value  for  cumulative  expenditures  and 
let  this  value  be  equal  to  the  cumulative  percent  of  budget  fig¬ 
ure  which,  is  associated  with  the  inflection  point  on  the  general 
curve. 


(3)  Using  the  relationships  established  in  1  and  2,  the 
top  half  of  the  general  curve  may  now  be  converted  from  cumula¬ 
tive  percentage  figures  to  forecast  cumulative  expenditures  for 
the  program  being  investigated. 

This  forecasting  method  has  several  advantages:  ’ 

(1)  The  time  over  which  the  program  is  planned  to  run' is 
taken  as  a  given  unless  evidence  to  the  contrary  is  discovered. 

(2)  The  lower  curve  forecast  is  mapped  into  the  general 
curve  format,  thereby  creating  a  smooth  S-shapcd  curve  for  the 
entire  program.  Simply  splicing  the  curves  as  is  done  with  the 
budget  curve  in  the  previous  section  will  often  create  discon¬ 
tinuities  in  the  curve. 

(3)  The  forecast  which  is  created  in  this  method  is 
based  strictly  on  the  assumption  that  expenditures  in  this  par¬ 
ticular  program  are  proceeding  in  the  same  manner  that  all  past 
programs  have  proceeded. 
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Th e  Wo r r. t  Possible  Cos t . 


Developing  the  forecast  for  the  worst  possible  cost  is  only  a 
matter  of  slightly  modifying  the  previous  most  likely  cost  fore¬ 
cast.  A  confidence  interval  for  the  most  likely  cost  is  calcu¬ 
lated  by  standard  statistical  methods.  The  upper  limit  of  this 
confidence  band,  based  on  whatever  level  of  confidence  was 
selected  by  the  analyst,  will  give  the  cost  figure  that  one  can 
be  X%  certain  will  not  be  exceeded.  Coupled  with  the  most  likely 
cost,  this  is  an  excellent  management  tool. 

rn  summary,  one  may  develop  three  possible  forecasts  from  the 
S-shaped  curve.  The  "spliced"  curve  forecast  using  the  program 
budget  curve  reflects  an  expenditure  of  the  lowest  possible  mag¬ 
nitude.  For  this  expenditure  to  be  realized,  the  program  must 
run  exactly  as  planned  from  the  inflection  point  onward.  This  is 
a  highly  unlikely  situation  if  any  increased  expenditures  have 
been  incurred  early  in  the  program.  The  most  likely  cost  and  its 
confidence  b.md  which  extends  to  the  upper  confidence  limit  (or 
the  worst-  possible  cost)  for  the  program  are  clearly  the  most 
realistic  forecasts.  This  is  because  the  method  of  mapping  the 
new  forecast  for  the  bottom  of  the  curve  into  the  general  curve 
format  places  the  entire  program  in  a  more  legitimate,  historical 
perspective. 


IV.  A  BENEFIT-COST  MODEL 


Benefit-cost  models  provide  an  approach  to  solving  problems 
of  choice.  In  this  case,  the  objective  might  be  to  choose  the 
aircraft  EMP  protection  design  that  provides  the  highest  ratio  of 
protection  achieved  per  dollar  spent.  Other  criteria  besides  a 
benefit-cost  ratio  might  also  be  appropriate.  For  example,  cri¬ 
teria  such  as  the  magnitude  of  first-year  costs,  budget  limita¬ 
tions,  uncertainty  about  future  inflation  or  discount  rates,  and 
the  degree  of  flexibility  inherent  in  a  particular  design  might 
be  important  for  a  particular  program.  A  good  benefit-cost  model 
should  permit  the  program  manager  to  consider  a  broad  range  of 
decision  criteria. 

Benefits  are  often  measured  in  dollars.  In  the  case  of  EMP 
aircraft  protection  however,  a  dollar  value  cannot  be  placed  upon 
the  amount  of  protection  achieved  from  a  particular  design.  We 
therefore  must  resort  to  some  nan-monetary ,  generic  measure  of 
the  amount  of  EMP  protection  achieved.  As  I  indicated  earlier, 
the  legitimate  application  of  a  benefit-cost  model  is  crucially 
dependent  upon  the  availability  of  such  a  measure. 

Keep  in  mind  then,  that  for  a  particular  EMP  protection 
design,  all  dollar  amounts  are  costs.  Benefit-cost  ratios  are 
achieved  by  placing  the  measure  of  che  amount  of  protection 
achieved  in  the  numerator  and  the  present  value  of  life  cycle 
costs  in  the  denominator. 

Let's  consider  the  time  value  of  money.  A  dollar  paid  today 
is  not  worth  a  dollar  tomorrow  because  there  is  an  opportunity 
cost  that  is  determined  by  the  amount  of  interest  a  dollar  could 
have  earned  in  an  alternative  investment.  For  example,  a  govern¬ 
ment  tax  dollar  today  is  not  worth  a  dollar  tomorrow;  it  is  worth- 
more  because  it  could  be  invested  in  the  private  sector  and  then 
be  reclaimed  with  interest  when  tomorrow  arrives.  Also, 
tomorrow's  dollar  is  not  worth  a  dollar  today.  After  all,  the 
government  could  deposit  90  cents  in  a  10  percent  investment  and 
receive  about  one  dollar  one  year  from  now.  Many  decision  prob¬ 
lems  deal  with  situations  in  which  amounts  of  money  that  exist  in 
different  time  periods  must  be  compared.  This  is  the  essence  of 
the  time  value  of  money  problem. 


tioru 


Consider  the  following  tools  for  time  value  of  money  caleula- 

(1)  Future  value,  single  amount: 

sn  -  S„(l  *  r)n 


where : 


Sn  -  Future  value  at  the  end  of  the  nth  period 
SQ  =  Present  value  at  time  zero 

r  =  Interest  (discount)  rate  expressed  as  a  decimal 
n  =  Number  of  periods 

( 2 )  Present  value,  single  amount: 

S„ 


So  " 


(1  +  *T 


where  the  definitions  expressed  above  remain  true. 

(  3 )  Present  value,  multiple  cash  amounts  over  t  i  m  e : 
(a)  Time  line: 


Sl,  S2. 


+  Cash  Flows  ^1 

j 

! 

r 

0  12] 

i  •«  s 

“  Cash  Flows 

(b)  The  present 
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•  »  sn_l,  ,  can  be  expressed  as: 
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^  n  -  2  ■  _ : 
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(c)  Equation  (3)  converts  the  cash  amounts  over  time 
as  expressed  on  the  time  line  in  paragraph  3  above  to: 


S; 


i=0  (1  +  r) 1 


0  1 


n- 1 
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H)  if  alternate  aircraft  EMP  pr oteet ion  uesigns  arc-  to 
bo  compared,  the  cash  amounts  per  time  period  for  each  alterna¬ 
tive  must  be  converted  to  sums  which  occur  at  a  single  point. 
For  example. 


Alternative  - - £ - * - “ 

A  lit 


Altcmat  ive 
Q 


lilt 


,_i _ 1 

n-2  n-1  n 


0  1  2.  3  4  5  /  n~2  rv^l  n 
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(b)  The  costs  which  occur  over  the  life  cycle  of 
Alternative  A  can  be  converted  to  a  present  value  cost  by  using 
equation  (3)  above.  A  similar  calculation  would  be  accomplished 
for  Alternative  B.  The  results  would  be  as  depicted  on  the  time 
lines  shown  below. 


Proaunt  Valuu 


Present  Value 


(5)  Benefit-cost  ratios  for  each  alternative  can  now  be 
calculated  using  the  measure  of  the  degree  of  protection  achieved 
that  I  discussed  earlier  and  the  present  value  cost  for  the 
alternative  in  question.  For  example: 

_ _ _  Measure  of  Protection  for  Alternative  A 

Benefit-Cost  Ratio  -  pr£-sent  Value  Cos  tT  for  Alternative  A" 


(6)  Let  s  consider  Uniform  Annual  Amounts  (sometimes 
referred  to  as  Annuities)  over  a  period  of  n  years.  A  shorthand 
version  of  the  present  value  cost  equation  can  be  used  to  sim¬ 
plify  the  analysis. 

(a)  Consider  our  present  value  equation  with  all  of 
the  equal  to  amount  A: 

n  a 

P.V.  =  (present  value)  =  - u — -  (4) 

i=l  (1  +C)1 

(b)  Since  A  is  independent  of  period  i,  we  can 

write: 

n  i 

P.V.  =  A  2  - ± - r- 

i=l  (1  +  r)1 

(c)  Which  can  be  shown  to  be  equal  to: 

P.V.  =  A  II  ±r)  n— 

r  (1  +  r)n 

As  a  final  note,  it  is  sometimes  useful  to  abbreviate  as  -fol¬ 
lows  : 

pvf  =  present  value  factor  *  - - - 

(1  +  r)n 

pvaf  =  present  value  of  an  annuity  factor  «  *  g« - =— ^ 

r  (1  +  r)n 


Now  consider  the  interest  (discount)  rate  used  in  government 
present  value  calculations.  The  choice  of  a  discount  rate  is 
based  on  the  premise  that  no  government  investment  should  be 
undertaken  without  explicitly  considering  the  alternative  use  of 
the  funds  which  it  absorbs  or  displaces. 

One  way  for  the  government  to  assure  this  is  to  adopt  a 
discount  rate  policy  which  reflects  private  sector  investment 
opportunities  foregone.  The  discount  rate  reflects  the  prefer¬ 
ence  for  current  and  future  money  sacrifices  that  the  public 
exhibits  in  non-government  transactions.  A  10  percent  rate  is 
considered  to  be  the  most  representative  overall  rate  at  the 
present  time.  The  government  prescribed  discount  rate  of  10  per¬ 
cent  represents  an  estimate  of  the  average  rate  of  return  on 
private  investment  before  corporate  taxes  and  after  adjusting  for 


a  test  at  other 


inflation.  The  cost  analysis  may  include 
discount  rates. 

The  economic  lives  of  alternative  aircraft  EMP  protection 
designs  govern  the  time  period  to  be  covered  by  a  program  evalua¬ 
tion.  Normally,  these  lives  will  approximate  the  life  of  the 
facility  protected.  The  economic  lives  for  the  alternatives 
should  be  set,  whenever  possible,  so  that  the  alternatives  yield 
benefits  (EMP  protection)  for  the  same  period  of  time.  If  this 
is  not  possible,  the  time  period  of  the  analysis  should  be  based 
on  the  life  of  the  asset  with  the  shorter  time  period.  In  this 
case,  the  residual  value  of  the  asset  with  the  longer  economic 
life  must  be  considered  in  computing  the  costs  of  that  alterna¬ 
tive  . 


Estimates  for  inflation  in  future  years  are  often  important 
in  program  evaluations.  To  detect  the  effect  of  changes  in  the 
purchasing  power  of  the  dollar,  the  program  manager  should  con¬ 
sider  both  constant  dollars  (without,  inflation)  and  current  dol¬ 
lars  (with  inflation)  in  analyzing  and  evaluating  alternatives. 
To  assure  consistency,  the  first  estimate  of  costs  for  each  year 
of  the  planning  period  should  be  made  in  terms  of  constant  dol¬ 
lars  (that  is,  in  terms  of  the  general  purchasing  power  of  the 
dollar  at  the  time  of  decision)  .  If  inflation  is  an  important 
factor  for  the  future,  a  second  computation  should  be  made  in 
terms  of  current  (inflated)  dollars.  When  there  is  reason  to 
believe  that  price  levels  will  significantly  affect  the  -choice 
between  alternatives,  the  indices  cited  earlier  should  be  \ised. 
When  including  inflation  for  a  cost  which  occurs  more  than  4 
years  beyond  the  present  year,  be  aware  of  the  uncertainty  in 
making  a  valid  economic  forecast,  and  the  fact  that  imputed 
values  for  inflation  may  change  considerably. 

To  determine  the  change  in  real  price  (exclusive  of  the 
effect  of  discounting)  ,  calculate  the  effect  of  inflation  -in. 
three  distinct  steps,  as  follows: 

(1)  Determine  the  constant  dollar  annual  costs  of  the 
alternative. 

(2)  Inflate  the  annual  cost  using  appropriate  indices. 

(3)  Apply  the  discount  rate  to  the  escalated  (current 
dollar)  amount. 


27 


The  present  value  equation  presented  earlier  can  also  be 
adjusted  Cor  uncertainty  with  regard  to  the  actual  amounts  of 
future  costs.  By  substituting  certainty  equivalents  for  expected 
future  costs,  the  model  permits  a  decision  maker  to  make  an 
explicit  tradeoff  between  the  expected  value  of  each  cash  amount 
and  its  associated  uncertainty,  or  risj'.. 

The  essential  characteristics  of  the  risk  adjusted  present 
value  equation  are  as  follows: 


PV  =  2 - r 

i=0  (1  +  r) 1 

where:  PV  ~  The  present  value  at  time  zero  of  a  series  of  risk- 

adjusted  cash  amounts  which  occur  in  the  future  for 
a  particular  program. 

v  -  An  appropriate  risk-free  discount  rate.4 

S*  ~  The  risk-adjusted  expected  value  of  the  cash  amount 
1  for  period  i ,  i  =  0 ,  1 ,  2 ,  .  .  .  n .  This  amount  is 
commonly  called  a  certainty  equivalent. 

Per  period  i,  a  certainty  equivalent  (S.)  can  be  obtained  by 
having  the  decision  maker  specify  the  amourrt  of  money  that*  would 
make  him  indifferent  between  this  certain  amount  and  the  expected 


3I  use  the  terms  "risk"  and  "uncertainty"  interchangeably. 
For  cither  term,  1  assume  that  future  cash  amounts  have  associat¬ 
ed  probability  distributions.  Risk  (uncertainty)  can  be  measured 
in  terms  of  the  degree  of  dispersion  about  the  mean  of  the  proba 
bility  distribution.  Also  note  that  the  probability  distribu-. 
tions  associated  with  future  cash  amounts  are  determined  by  the 
uncertainties  inherent  in  the  development,  production,  operation, 

and  maintenance  of  a  particular  .EMP  protection  design.  As  dis¬ 
cussed  earlier,  factors  for  inflation  due  to  changing  resource 
costs  can  also  be  included.  However ,  larger  risks,  such  as  the 
risk  associated  with  the  stability  of  the  monetary  system,  are 
exogenous  to  the  model. 


4In  some  versions  of  the  model,  the  discount  rate  is  adjust¬ 
ed  to  include  a  risk  premium  for  each  period.  This  adjustment  is 
used  in  lieu  of  the  certainty  equivalent  adjustments  to  the  ex¬ 
pected  cash  amounts.  Again,  the  d<V9v,roe^  0f  ioi™£nt%nd ^roduc- 
solely  by  uncertainties  inherent  in  tho  development  and  produc 

tion  of  that  particular  weapon  system. 
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cash  amount  with  its  associated  risk.  Thu  magnitude  of  this  cer¬ 
tainty  equivalent  is  determined  by  the  decision  maker's  attitude 
toward  the  risk.  There  are  undoubtedly  some  decision  makers  who 
would  proier  risk  and  some  who  may  bo  indifferent  to  risk,  but 
conventional  opinion  among  economists  holds  that  the  majority  of 
decision  makers  involved  with  large  sums  of  money  tend  to  bo 
r  isk-aver.se  . 

Thus,  each  S*  is  calculated  by  multiplying  the  expected  cash 
amount  for  period  i  by  a  certainty  equivalent  factor  which  is 
based  upon  the  decision  maker's  attitude  toward  risk.  The  cer¬ 
tainty  equivalent  factor  for  a  cost  must  be  a  number  greater  than 
one,  i.e.,  the  present  value  cost  is  made  larger. 


V.  SUMMARY  AND  RECOMMENDATIONS 


The  C-130  testbed  program  will  generate  valuable  cost  data 
which  can  be  used  to  develop  planning  models  for  future  aircraft 
EMP  protection  programs.  In  this  paper,  I  have  briefly  disussed 
a  life  cycle  cost  model,  presented  an  S-curve  model  for  R&D  cost 
forecasting,  and  finally,  discussed  a  possible  benefit-cost  model 
for  comparing  alternative  aircraft  EMP  protection  designs.  The 
S-curve  model  can  be  developed  from  the  life  cycle  cost  data  base 
with  no  limiting  requirements.  The  benefit-cost  model  cannot  be 
developed  unless  it  includes  some  generic  measure  of  the  degree 
of  protection  achieved  by  the  alternate  EMP  protection  designs. 

The  life  cycle  cost  model  should  be  coordinated  with  all 
agencies  involved  in  the  C-130  testbed  program.  Changes  should 
be  made,  if  appropriate,  and  cost,  data  should  be  collected. 
Additional  cost  data  on  previous  aircraft  EMP  protection  programs 
should  also  be  gathered,  and  as  accurately  as  possible, 
integrated  into  the  model.  Please  note  that  the  cost  data  and 
milestones  for  each  program  will  be  kept  separate  within  the 
model. 

I  recommend  that  we  continue  to  explore  possible  uses  for  the 
life  cycle  cost  data  base.  The  S-curve  model  and  the  be-nefit- 
cost  model  appear  to  be  useful  tools  for  future  EMP-pr otect ion 
planners.  Other  models  are  possible.  Given  the  amount  of  atten¬ 
tion  focused  on  defense  budgets  and  the  relatively  large  expendi¬ 
tures  envisioned  for  a  large-scale  aircraft  EMP  protection 
effort,  it  would  appear  worthwhile  to  provide  EMP  protection 
planners  with  appropriate  models  for  conscientious,  and  accurate 
budget  forecasts. 
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